Phase 3.5 — Cooperative atto Audit and MailSite Daemon

Arc Post-Mortem Summary. A direct follow-on to Phase 3's cooperative web server. It audited each atto server for the same single-process blocking pattern and carried the non-blocking treatment across the rest of atto, and it made the MailSite daemon's own loop cooperative. Delivered in slices under the constraints carried in from Phase 3.

Arc started 2026-06-20, straight after the Phase 3 cooperative server work landed. Phase 3 made the atto web server non-blocking: long work (an external IMAP probe, a password hash, a big media list, an SMTP send, a local-mailbox read) now yields to the event loop and is re-entered, instead of freezing every other web connection. This arc carries the same treatment to the rest of atto — it audits each server for the same single-process blocking pattern, makes the MailSite daemon's own loop cooperative, and writes worked examples of the fiber primitives so later code follows the pattern rather than reinventing it.

Audit of the atto servers (verified 2026-06-20)

Every atto server is a non-blocking reactor: sockets are non-blocking, a select loop drives reads and writes, partial input is buffered across passes. So a slow peer never blocks the loop. The freeze instead comes from a command handler doing blocking work mid-pass — a store lock, a file read, an upstream lookup. The fix is the web-side one: run the handler in a fiber so its cooperative blocking op yields.

  1. WebSite.php — done. Handlers run in fibers (Phase 3) that suspend on store reads, IMAP, the TLS handshake, CPU yields, and worker pipes.
  2. MailSite.php — this arc. Reactor network path is fine; processOne handlers block on the file store (flock and disk). Slice 7 made the store read locks cooperative; this arc runs processOne in a fiber so they yield, then does the same for the delivery write locks.
  3. SshSite, FtpSite, GopherSite, DnsSite (TCP) — same pattern, atto repo. All reactors; handlers block on file reads (Gopher, FTP retrieve), the FTP data-channel stream_socket_client connects, or DNS upstream lookups. Not vendored in Yioop, so they get the same minimal fiber frame in the atto repo, following this arc's examples.
  4. TurnSite, H3Listener, H3QuicheListener, DnsSite (UDP) — out of scope. UDP / QUIC and connectionless: no per-connection accept or blocking read, so the freeze pattern does not apply. Listed so the audit is complete.

Slices

  1. ✓ Slice 1 — run the daemon's command handling in a fiber. readClient now runs a connection's processOne loop in a fiber; if a store lock yields, the fiber parks and the loop resumes it each pass (polling while any are parked), dropping it on finish or shutdown. With Slice 7's cooperative read locks, the daemon's store reads now yield instead of freezing every other mail connection. The scheduler question is settled: each server stays self-contained, so this is a small inline frame, not a shared class.
  2. ✓ Slice 2 — cooperative delivery write locks. The daemon's exclusive-lock write paths (allocUid, bumpFolderUidNext), left blocking in Slice 7, now use the cooperative wait too, so a delivery waiting on a reader yields rather than stalling the loop. cooperativeFlock now returns a bool and tells contention (retry) from a real lock error (fail fast), so the write paths keep their failure handling. Mirrored to the atto repo's MailSite (Slice 1 and 2 together).
  3. ✓ Slice 3 — the atto-only servers (atto repo). Of the four, only two have a blocking handler: FtpSite's data-channel stream_socket_client connect, and GopherSite's file_get_contents read-all of a served file. Each gets the command-fiber frame plus a cooperative version of that one op (async connect; chunked read with a yield). DnsSite (UDP datagram) and SshSite (reactor reads, one startup host-key read) have no per-request blocking, so they are left as is. Done one server per step; not unit-testable in the Yioop sandbox, so these lean on integration testing. Done: FtpSite got the command-fiber frame plus cooperative accept/connect on the data channel (3a) and cooperative transfer pumps for RETR/STOR/LIST (3b); GopherSite's read-all became a chunked cooperative read, validated byte-identical against a real lynx client; DnsSite and SshSite were confirmed to have no per-request blocking and left as is.
  4. ✓ Slice 4 — worked examples of the fiber primitives. A short runnable reference for the few primitives this work introduced — the readable/writable suspend, the plain yield, the cooperative lock, the command-fiber frame, the worker offload — added as a new atto example, with a defer case in the benchmark example to show the speed-up. Done: example 26 ships as both a CLI run and a web page whose per-demo code panels are filled by reflection at request time (so they cannot drift from the real source); the benchmark example carries the offload defer case. Closes the arc.

Arc closed 2026-06-21. All four slices landed. The atto servers that serve real requests (Web, Mail, Ftp, Gopher) now cooperate inside a fiber, while the daemon run on its own, the command-line tools, and Apache stay byte-for-byte unchanged. Dns and Ssh were audited and need no per-request change.

Constraints carried in

  1. atto files stay self-contained: no new files in atto_servers/, no reaching into another atto file, and the daemon defines its own constants rather than reading Yioop Config.
  2. Every change keeps the non-fiber path (the daemon run on its own, the command-line tools, Apache) byte-for-byte as it was: cooperation only engages inside a fiber.
  3. The full suite stays green at each step; new primitives get a focused test the way the web-side ones did.